Failed Interop Tests

QUIC Interop Runner, client vs. server

• aioquic vs. neqo-latest: A
• go-x-net vs. neqo-latest: U A
• kwik vs. neqo-latest: A
• lsquic vs. neqo-latest: R A
• msquic vs. neqo-latest: Z A
• mvfst vs. neqo-latest: Z 3 A L1 C1
• neqo vs. neqo-latest: Z A
• neqo-latest vs. aioquic: Z C1
• neqo-latest vs. lsquic: C1
• neqo-latest vs. msquic: R Z A L1 C1
• neqo-latest vs. mvfst: DC B U A L1 L2 C1 C2
• neqo-latest vs. neqo: A
• neqo-latest vs. neqo-latest: Z A
• neqo-latest vs. quic-go: Z
• neqo-latest vs. quiche: L1 C1
• neqo-latest vs. quinn: E A
• neqo-latest vs. s2n-quic: R
• neqo-latest vs. xquic: Z A L1
• ngtcp2 vs. neqo-latest: A
• picoquic vs. neqo-latest: R A
• quic-go vs. neqo-latest: A
• quiche vs. neqo-latest: R 3 A
• quinn vs. neqo-latest: R Z E A
• s2n-quic vs. neqo-latest: E A L1
• xquic vs. neqo-latest: M A

Benchmark results

Performance differences relative to 7070e7b.

• drain a timer quickly time: [310.01 ns 318.49 ns 326.46 ns]
  change: [-1.9586% +0.3007% +2.5837%] (p = 0.79 > 0.05)
  No change in performance detected.

• coalesce_acked_from_zero 1+1 entries
  time: [194.35 ns 194.89 ns 195.43 ns]
  change: [+0.1564% +0.6558% +1.2481%] (p = 0.02 < 0.05)
  Change within noise threshold.

• coalesce_acked_from_zero 3+1 entries
  time: [236.97 ns 237.57 ns 238.21 ns]
  change: [-0.4035% +0.0992% +0.5474%] (p = 0.70 > 0.05)
  No change in performance detected.

• coalesce_acked_from_zero 10+1 entries
  time: [236.02 ns 236.94 ns 237.98 ns]
  change: [-0.9947% -0.2489% +0.3826%] (p = 0.51 > 0.05)
  No change in performance detected.

• coalesce_acked_from_zero 1000+1 entries
  time: [217.25 ns 218.55 ns 221.27 ns]
  change: [-0.4078% +14.797% +43.667%] (p = 0.31 > 0.05)
  No change in performance detected.

• RxStreamOrderer::inbound_frame()
  time: [118.12 ms 118.21 ms 118.31 ms]
  change: [-3.5051% -3.4102% -3.3198%] (p = 0.00 < 0.05)
  💚 Performance has improved.

• transfer/Run multiple transfers with varying seeds
  time: [117.99 ms 118.29 ms 118.59 ms]
  thrpt: [33.730 MiB/s 33.815 MiB/s 33.902 MiB/s]
  change:
  time: [+0.1950% +0.5033% +0.8290%] (p = 0.00 < 0.05)
  thrpt: [-0.8222% -0.5008% -0.1946%]
  Change within noise threshold.

• transfer/Run multiple transfers with the same seed
  time: [118.06 ms 118.29 ms 118.51 ms]
  thrpt: [33.751 MiB/s 33.816 MiB/s 33.880 MiB/s]
  change:
  time: [+0.2335% +0.4897% +0.7551%] (p = 0.00 < 0.05)
  thrpt: [-0.7494% -0.4873% -0.2330%]
  Change within noise threshold.

• 1-conn/1-100mb-resp (aka. Download)/client
  time: [1.1112 s 1.1183 s 1.1273 s]
  thrpt: [88.710 MiB/s 89.422 MiB/s 89.995 MiB/s]
  change:
  time: [-0.0438% +1.9295% +3.5391%] (p = 0.05 > 0.05)
  thrpt: [-3.4181% -1.8930% +0.0438%]
  No change in performance detected.

• 1-conn/10_000-parallel-1b-resp (aka. RPS)/client
  time: [427.16 ms 429.48 ms 431.80 ms]
  thrpt: [23.159 Kelem/s 23.284 Kelem/s 23.410 Kelem/s]
  change:
  time: [-0.3718% +0.4094% +1.1964%] (p = 0.32 > 0.05)
  thrpt: [-1.1823% -0.4077% +0.3732%]
  No change in performance detected.

• 1-conn/1-1b-resp (aka. HPS)/client
  time: [48.916 ms 49.296 ms 49.671 ms]
  thrpt: [20.132 elem/s 20.286 elem/s 20.443 elem/s]
  change:
  time: [+0.1278% +1.1881% +2.2877%] (p = 0.03 < 0.05)
  thrpt: [-2.2366% -1.1741% -0.1276%]
  Change within noise threshold.

Client/server transfer results

Transfer of 134217728 bytes over loopback.

⬇️ Download logs

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.25%. Comparing base (7070e7b) to head (92fd589).

@@           Coverage Diff           @@
##             main    #1867   +/-   ##
=======================================
  Coverage   93.25%   93.25%           
=======================================
  Files         110      110           
  Lines       35813    35813           
=======================================
  Hits        33397    33397           
  Misses       2416     2416           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.